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Chapter 37 

Program Evaluation and 
Outcomes Assessment 

Documenting the Worth of Educational 
Programs 

Bradley T Erford & Cheryl Moore-Thomas 



The impetus for the call for school reform and accountability may 
be found in the 1983 report A Nation at Risk (Angelo & Cross, 1993; 
Finn & Kanstroom, 2001). National concerns regarding quality 
education make program evaluation and outcomes assessment more 
important than ever. Traditionally, however, educators have failed to 
hold their programs and services accountable, or to provide evidence 
that selected activities were achieving intended results (Lombana, 1985). 

Some argue that the educational process is so complex that 
obtaining a true measure of its services and results is difficult at best. 
Others note that school staffs are so busy meeting the needs of students 
that they shift the time that should be spent on evaluation to instruction 
and programming. Others suggest educational practitioners lack 
understanding of the methodology and procedures of accountability 
studies. Whatever the reason, the result is a lack of accountability that 
threatens the success of students and the future of some educational 
institutions. Each reason contributes to avoidance of professional and 
ethical obligations to ensure that educational programs are of high 
quality and are effective in meeting students’ needs. Without 
accountability, education service providers and the greater education 
community may be regarded as suspect. 

Stone and Bradley (1994) suggest six purposes of evaluation: (a) 
to measure the effectiveness of a total program and its activities; (b) to 
collect data that will help determine what program modifications are 
needed; (c) to determine the level of program acceptance by and support 
from stakeholders; (d) to obtain information that can be used to inform 
the public; (e) to collect data that add to staff evaluation; and (f) to 
analyze the program budget and compare expenditures to future program 
needs. In general, then, program evaluation and outcomes assessment 
work to document and determine the worth of the entire school program. 
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The purpose of this chapter is to give educators the basic tools 
needed to design and conduct individualized, effective outcomes 
assessment and program evaluation that will aim to document and 
determine the worth of specific educational programs. 

The Assessment Loop 

The evaluation of a comprehensive educational program is at least 
two tiered. Gysbers and Henderson (2000) describe two key elements: 
program evaluation (process), and results (outcomes) evaluation. 
“Program evaluation is the process of systematically determining the 
quality of a school program and how the program can be improved” 
(Sanders, 1992, p. 3). Program evaluation can also be thought of as a 
process analogous to the measurement concept of content validity 
(Gysbers & Henderson, 2000). Content validity is determined by a 
systematic examination of a test’s, or in this case program’s, content. 
In the context of educational program evaluation, an important guiding 
question emerges: Does the school have a written, comprehensive 
program that is fully implemented and aligned with district, state, or 
national standards? Outcomes evaluation, on the other hand, attempts 
to answer the following question: Does the educational program in fact 
produce the intended outcomes (Terenzini, 1989)? 

Practical guidelines are essential to conducting effective program 
evaluations and outcomes assessment (Atkinson, Furlong, & Janoff, 
1979; Fairchild, 1986; Krumboltz, 1974). It is most practical to connect 
program concerns to only one or two clearly articulated and defined 
questions. This practice may help to ensure focused and manageable 
assessment. 

There is much confusion regarding what program evaluation and 
outcomes assessment are and are not. Undoubtedly, the assessment 
process is systematic, ongoing, and cyclical. The program evaluation 
and outcomes assessment processes start small and build upon what is 
found to work. Successful methods and goals of individual programs 
are determined and replicated so that over time necessary program 
refinements work to build a comprehensive educational program that 
impressively meets an institution’s mission. The assessment loop 
presented in Figure 1 provides a way to visually conceptualize program 
evaluation and the ways in which outcome studies can be used to 
improve educational programs. 
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Figure 1. Program Evaluation Cycle 
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Many educators view assessment as a discrete component of 
education; however, assessment is actually an integrated part of a 
continual process for program improvement. Assessment procedures 
begin with an institution’s mission. The mission ideally permeates every 
aspect of the educational institution. A school’s mission should be 
evident in its structure, decision-making processes, interpersonal 
interactions, programmatic regularities, and behavioral regularities. The 
institutional mission provides the basis from which meaningful, 
institution- specific assessment questions will arise. These assessment 
questions lead to the determination of what evidence must be collected. 
Evidence can provide crucial information about program evaluation 
and program results (i.e., outcomes). 

Evidence is typically derived from standardized or informal 
measures, student performances, or student products. Once a school 
has gathered evidence, it must interpret the data, then draw conclusions 
regarding the educational program’s worth, strengths, weaknesses, and 
outcomes. A school should use these interpretations and conclusions to 
change the entire program or to improve parts of the program. 
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As assessment information is used to prompt programmatic 
changes, goal setting and the posing of new questions begins again. 
The loop in Figure 1 should never stop. It represents a continuous 
process in which assessment results are interpreted and fed back into 
the improvement process. Unless assessment involves all components 
of the assessment loop, program evaluation efforts may prove futile or 
incomplete. With this understanding of program evaluation and 
outcomes assessment in mind, a school can consider specific definitions 
and processes. 



Assessment Terms and Processes 

A number of terms associated with outcomes assessment and 
program evaluation are important to understand. Evaluation is the 
measurement of worth and indicates that a judgment will be made 
regarding the effectiveness of a program (Cronbach, 1983). Specificity 
is key to evaluation. Clearly stating what is to be measured and how it 
is to be measured is fundamental to meaningful, effective assessment. 

Evidence is qualitative or quantitative data that help make 
judgments or decisions. Evidence can be gleaned from a number of 
sources, including portfolios, performances, external judges or 
examiners, observations, local tests, purchased tests, student self- 
assessments, surveys, interviews, focus groups, and student work. Some 
of this evidence may already be routinely collected by a school or district 
and may thus provide a readily accessible source of data about program 
effectiveness. Which evidence source to use, however, is determined 
by the specific question to be answered. Evidence selection should be 
made carefully. If measures are used, they must be reliable and valid. 
Sometimes ineffective program outcomes stem exclusively from poor 
or inappropriate measurement choices rather than program deficits. 

Formative evaluation is evaluative feedback that occurs during 
the implementation of a program. Summative evaluation is feedback 
collected at a specified endpoint in an evaluation process (Worthen, 
Sanders, & Fitzpatrick, 1997). Formative evaluation allows for 
midcourse corrective action. Although summative evaluation is more 
widely used, formative evaluation is an advantageous endeavor when 
the time, dollar, and human cost of educational programming is 
considered. 

A stakeholder is anyone involved or interested in, or potentially 
benefiting from, a program (Sanders, 1992; Worthen et al., 1997). 
Students, parents, teachers, school counselors, administrators. 
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community leaders, college faculty, and local employers, among others, 
are potential educational stakeholders. Inclusion of a variety of relevant 
stakeholders is important to the assessment process. 

Reporting Assessment Results 

Reporting assessment findings is also important to effective 
program evaluation and outcomes assessment. The school leadership 
team — including administrators, teachers, professional school 
counselors, staff members, parents, and other appropriate stakeholders — 
should write and be involved in every step of the reporting process. 
Although a comprehensive report may be helpful for analysis purposes, 
a one- to two-page executive summary should also be prepared for 
release to system administrators and the general school community. As 
general guidelines for report writing, Heppner, Kivlighan and Wampold 
(1992) suggest, “(1) be informative, (2) be forthright, (3) do not overstate 
or exaggerate, (4) be logical and organized, (5) have some style, (6) 
write and rewrite, and (7) when all else fails, just write!” (p. 376). 
Although they may have intended the last guideline to be humorous, it 
is important to note that results must be documented, interpreted, and 
reported for accountability to occur. 

Case Study 

The following case integrates, in a practical way, the key concepts 
of outcome assessment and program evaluation presented in this chapter. 
Beall Middle School is in a suburban middle-class community. Over 
the past 10 years, the student population of Beall has changed to match 
the changing demographics of the surrounding community. Beall’s 
current 600-student enrollment is 31 percent minority. Beall enjoys a 
98 percent daily attendance rate. Three percent of the students receive 
free or reduced-cost lunch. 

The slogan of Beall Middle School is “Success for Every Student.” 
The staff has embraced the middle school philosophy to deliver a 
comprehensive, stable program that provides educational opportunity 
for all students. The leadership team at Beall has been closely monitoring 
test results. Over the past four years Beall’s standardized test scores 
have fallen significantly. In light of this finding, the leadership team 
posed several educational questions, including Is the current 
instructional program enabling all students to meet the school’s mission 
of success? From the initial question, the school counselor posed a 
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related discipline-specific question aligned with the school’s mission, 
the district guidelines, and the National Standards for School Counseling 
Programs. The counselor asked. Are all students acquiring “the attitudes, 
knowledge, and skills that contribute to effective learning in school 
and across the life span?” (Campbell & Dahir, 1997, p. 20). The school 
leadership team felt that the discipline-specific question was on target 
and focused enough to guide meaningful assessment and evaluation. 

Having identified their educational question, the counseling 
department and leadership team determined that surveys would provide 
an appropriate source of evidence to answer the school’s question. The 
department then developed student, staff, and parent surveys based on 
the school district’s guidance and counseling curriculum and the 
competencies of the academic development standard of the 

National Standards for School Counseling Programs (Campbell 
& Dahir, 1997). To help ensure validity, central office guidance and 
counseling staff, a college school counseling faculty partner, and parent 
and student focus groups reviewed the surveys. Distributed, completed, 
and tallied over the next two months, the surveys yielded important 
data. 

The results seemed to suggest that significant percentages of the 
students and parents wanted more help with time-management skills. 
The teacher surveys seemed to suggest that students needed more help 
identifying attitudes and behaviors that lead to successful learning. The 
counseling department and leadership team wrote a brief, user-friendly 
report of the data to be shared at leadership, staff, and PTA meetings. 
Using these data, the counseling department, in collaboration with the 
instructional team leaders and chairs of the academic departments, 
redesigned the second semester comprehensive guidance and counseling 
plan to include emphasis on these areas. A key component of the 
redesigned plan called for formative evaluation and midcourse 
correction that would be jointly monitored by the school counselors, 
administrators, and representatives of the leadership team. Evidence to 
be examined for formative evaluation included students’ third quarter 
work-study skill grades, educational management team notes and 
strategies developed and implemented during the third quarter, and a 
student assessment built into the third quarter schoolwide guidance 
lessons on time management. All stakeholders were confident that this 
assessment loop, with its inherent level of accountability, would more 
effectively monitor and guide Beall Middle School to its intended goal 
of success for every student. 
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Summary 

The Beall Middle School case suggests that program evaluation 
and outcomes assessment are important tools for effective service 
delivery in every area of a school program. As calls for increased 
accountability in education rise, educators, school counselors, other 
school staff members, and other educational stakeholders will be well 
advised to implement ongoing, cyclical program evaluation and 
outcomes assessment processes. 
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